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Abstract 

A calculation of the fluxes of primary particles arriving to the Earth's vicin- 
ity as well as those produced in the interactions of the primaries with the atmo- 
sphere is presented. The result of calculations is compared with the experimental 
data obtained with the Alpha Magnetic Spectrometer (AMS). A good agreement 
of calculated and measured fluxes of charged particles supports the viability of the 
atmospheric neutrino flux calculation. 



1 Introduction 



The interpretation of the data on the atmospheric neutrinos obtained by different experi- 
ments [H 12 El in El strongly relies on the calculations of neutrino fluxes 00111 13 UHl HH 

^lEl^JEj. Starting from a common assumption that the primary cosmic rays arrive in 
the vicinity of the Earth isotropically the calculations differ in the level of sophistication 
used by different authors to describe the production of the secondaries in the hadronic 
interactions of primary cosmic particles (mostly protons) with the atmosphere and in the 
way the secondaries are propagated in the Earth's magnetic field. Different calculations 
producing considerably different results, the validity of neutrino flux calculations can only 
be checked by comparing the predictions of these calculations with the experimental data. 
Although the difference in the measured and calculated atmospheric fluxes of upgoing and 
downgoing neutrinos is considered as a clear evidence for neutrino oscillations it is 
nevertheless recognized that the same calculations [3 [T2j seem to fail to properly 
describe the measured east-west asymmetry of atmospheric neutrino fluxes jini related to 
a well known ^3 observation that due to the influence of the Earth's (nearly dipole) 
magnetic field the fluxes of the primary predominantly positive (p,a-particles) hadroni- 
cally interacting cosmic rays reaching the atmosphere contain more particles coming from 
the west than from the east. Thus, more accurate and detailed calculations describing 
propagation of primary cosmic rays and their interactions with the atmosphere is needed. 



The recent data from the Alpha Magnetic Spectrometer (AMS) fl^ 123 provide a 
vast amount of information on the fluxes of both primary and secondary particles detected 
in the near Earth orbit covering most of the Earth's surface. The AMS data display not 
only the common features of the magnetic cutoff dependence on the distance from the 
magnetic equator but also reveal the presence of particles well under the cutoff [24,, 25^ . 
These particles originate from the production and eventual decay of the secondaries pro- 
duced in the inelastic interactions of the primary cosmic particles with the atmosphere 
and consequently carry the information about neutrinos produced in decays of the sec- 
ondaries. A comparison of the results obtained in the calculations with the fluxes of the 
secondaries measured with AMS represents a means to prove or disprove the validity of 
the results of the calculations concerning the predictions of atmospheric neutrino fluxes. 

In the following the result of calculation of the near Earth fluxes of charged particles 
and atmospheric neutrinos is presented. First, the model used and its parameters is 
described. Section 3 describes the origin of the particles near the Earth. Section 4 is 
devoted to results of calculations of fluxes of charged particles at the altitude of 400 km 
from the Earth's surface corresponding to the AMS flight altitude. The simulated fluxes 
of charged particles are compared with those measured with AMS in section 5. Section 6 
is devoted to the results of calculations of atmospheric neutrino fluxes at the ground level 
in general. The neutrino fluxes at different experimental sites are discussed in section 7. 
The last section gives a summary of the results and draws some conclusions. 

2 Description of the model 

The calculation uses the Monte Carlo technique and is based on the GEANT3/GHEISHA 
package [SE]- This package is extensively used for the simulation of processes in various 
detectors of high energy physics experiments. To use this package for the simulation on a 
scale of propagation of cosmic rays in the Earth's magnetic field, their interactions with 
the Earth's atmosphere and the Earth itself, followed by production, propagation and 
decay of secondary particles, the parameters of the corresponding media are adapted to 
scale them from a thousands of kilometer scale of cosmic ray travel and interaction to a 
centimeter scale characteristic for GEANT based simulations. 

The Earth is modeled to be a sphere of 6378.14 km radius of a uniform density of 5.22 
g/cm^ derived from the total mass of the Earth. The atomic and nuclear properties of 
the Earth are taken to be those of Ge - the closest in density (5.32 g/cm^) element. The 
Earth's atmosphere is modeled by 1 km thick variable density layers of air extending up 
to 71 km from the Earth's surface. The density change with altitude is calculated using 
a Standard Atmosphere Calculator [27]. 

The Earth's magnetic field for the year 1998 is calculated according to World Magnetic 
Field Model (WMM-2000) j2H] with 6 degrees of spherical harmonics. The WMM-2000 
is a predictive model based on the magnetic field measurements on a global scale. 
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The flux ($) of liigli energy primary protons in tfie Solar system is parametrized on 
the basis of the existing experimental data [211 EHl (HDl IHH 1321 (HB] by a power low in 
rigidity (R), $(-R) ~ with 7=2.79. At the energies below several GeV the spectrum 
is corrected for the solar activity to be similar to that of the year 1976 - the year of 
a similar solar activity as in the year 1998 according to the 11 year solar cycle. Primary 
protons are isotropically emitted from a sphere of 10 Earth's radii. The long distance from 
the Earth of the emission point is sufficient for a particle to endure sufficient influence of 
the Earth's magnetic field to form an adequate topology of the flux in the Earth's vicinity. 

Secondary particles produced in the interactions with the atmosphere and the Earth 
are traced in the magnetic fleld following a normal GEANT interaction/decay procedure. 

3 Origin of particles near Earth 

The shape of magnetic fleld determines the behaviour of charged particles at their ap- 
proach to the Earth and the eventual motion of primaries and secondaries in the Earth's 
vicinity. The channeling of the particles in the Earth's magnetic fleld leads to a well known 
effect that the flux of particles coming from space is at its maximum at the magnetic poles 
(see top FigUl). This effect is associated with the magnetic cutoff of low momenta parti- 
cles in the equator region. A peculiar feature of particle channeling from the outer space 
is that most of the particles detected in the pole region in the Earth's vicinity have their 
origin in the equator region of the isotropically shining sphere at 10 Earth's radii used in 
the simulation (bottom Fig^. The efficiency of channeling of the particles depends on 
the emission parameters such as particle momentum and emission direction. 

Figure |21 shows the emission parameters of the particles reaching the Earth. One can 
see in figure |21 that small momentum particles even emitted in the direction opposite to 
the Earth can reach the Earth's vicinity whereas high momentum particles can do that 
only when the initial direction is close to a straight line between the emission point and 
the Earth. The dashed lines in figure |21 show the cuts on the emission parameters used 
in the simulation. An additional cut is set on the momentum of primary protons. It is 
set to be of 0.5 GeV/c for two reasons. First, due to the detector material thickness the 
minimum kinetic energy required for a proton to trigger AMS is ~ 0.13 GeV corresponding 
to a proton momentum of 0.5 GeV/c. Secondly, the kinetic energy threshold of charged 
pion production in p- nucleus collisions is ~ 0.16 GeV |SS] corresponding to a proton 
momentum of 0.54 GeV/c. Protons with energies below the threshold can not produce 
charged pions which represent the source of detectable secondaries and eventually of 
atmospheric neutrinos. The cut on the proton momentum eliminates the few events in 
the region forbidden by the cut of the bottom Figj21 
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Of all protons with the momentum spectrum described above uniformly emitted from 
the sphere of 10 Earth's radii only a 7.4 * 10~^ part reaches the Earth's atmosphere. In 
the subsequent simulation only particles with initial kinematics compatible with those 
reaching the Earth's atmosphere are traced the whole way until they interact with the 
atmosphere or the Earth itself or/and continue the journey beyond a sphere of 15 Earth's 
radii. The charged secondaries from the interactions are traced until they go below a limit 
of 0.125 GeV/c. 

Figure El shows an example of the trajectory of a particle reaching the Earth. (Note 
that the AMS flight altitude of 400 km corresponds to 1.06 Earth's radii). Due to influence 
of the Earth's magnetic field the direction of a particle at its entrance to the atmosphere 
is by far different from the initial direction. 

Primary cosmic particles may endure an elastic or inelastic scattering in the atmo- 
sphere thus changing their direction and/or momentum. Figure E] shows the trajectory of 
such a particle which eventually reenter the atmosphere but at a place far away from the 
point of initial interaction. As it will be shown later, there is a considerable amount of 
such particles in the total flux of incoming cosmic rays. 

Secondary particles produced in the inelastic interactions of primary cosmic rays with 
the atmosphere follow a usual pattern of hadronic or electromagnetic cascade and most 
of them quickly end up in the atmosphere itself or in the Earth. However, under the 
influence of the Earth's magnetic field some (predominantly stable) particles from the 
atmospheric showers escape upward from the atmosphere and travel long distances before 
being absorbed in the atmosphere. The trajectory of such particle depends on the kine- 
matics of the particle production and the geomagnetic coordinates of the origin. Unless 
they interact with the atmosphere the particles with the same rigidity (p/z) follow the 
same trajectory. Figure El shows the trajectory of a positive particle produced in the atmo- 
sphere in the northern hemisphere, reflected at a mirror point in the southern hemisphere 
before being absorbed at an approximatively the same latitude but a different longitude. 

The spiraling of the particles, their motion from and to the Earth's surface as well as 
their bouncing across and drifting along the magnetic equator is determined by the shape 
of the Earth's magnetic field. Depending on the production place and direction with 
respect to the Earth's surface secondary particles can make a return trip as illustrated in 
FigiSl a one way trip across the magnetic equator, a multiple bounce (FiglHl) or even a 
round the globe trip (FigEj). At the limit a secondary particle produced in the atmosphere 
over a certain limited region of the Earth (called South Atlantic Anomaly) can have a 
closed trajectory (FigjSl) and could live forever if it would not interact with the atmosphere 
at the place where it was created nor lose its energy on synchrotron radiation. 

As one can see in figures Ell, Etl, Htl andjHli the particles in their motion pass the level 
of the AMS flight (1.06 Earth's radii) going up and down several times. The geographic 
coordinates of the place over which a secondary particle can be detected at a certain 
altitude is connected with the place of origin of the particle. 
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Figure 1: Top: The points of impact of primary cosmic rays on the Earth. Bottom: The 
points of origin on the 10 Earth's radii sphere of primary cosmic rays reaching the Earth. 
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Figure 2: The phase space in cos (©emission) /momentum (top) and cos ( ©emission )/0emission 
(bottom) planes of particles reaching the Earth's vicinity. The dashed lines show the 
limits used in the simulation. 
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Primary particle trajectory. P=14 GeV/c 
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Figure 3: Primary proton trajectory in the Earth's magnetic field for a particle reaching 
the Earth, a) General view, b) View from 90 degree west c) View from the north pole, d) 
View at the Earth approach. 
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Scattered particle trajectory. P=9.03 GeV/c 
a) 




is 2 

2 1.5 






c) 


1 

W 0.5 























-0.5 








-1 




/ i \ \ ' 




-1.5 








-2 




1 1 1 1 1 1 1 







2 


U( 






1.8 














1.6 




1.4 




1.2 



-10 12 
Earth radii 




100 
Longitude 



Figure 4: Trajectory of proton scattered in the atmosphere, (a) General view, (b) View 
from 90 degree west, (c) View from the north pole, (d) View of the scattering point and 
the Earth approach. 
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Short path particle trajectory. P=1.5 GeV/c 
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Figure 5: Trajectory of secondary positive particle with short path length, (a) General 
view, (b) View from 90 degree west, (c) View from the north pole, (d) Altitude at 
different latitude. 
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Medium path particle trajectory. F=2.5 GeV/c 
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Figure 6: Trajectory of multi bouncing secondary particle with medium path length, (a) 
General view, (b) View from 90 degree west, (c) View from the north pole, (d) Altitude 
at different longitude. 
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Long path particle trajectory. P=2.0 GeV/c 




Earth radii Longitude 



Figure 7: Trajectory of multi bouncing secondary particle with long path length, (a) 
General view, (b) View from 90 degree west, (c) View from the north pole, (d) Altitude 
at different longitude. 
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Trapped particle trajectory. P=1.65 GeV/c 
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Figure 8: Closed orbit trajectory of multi bouncing secondary particle, (a) General view, 
(b) View from 90 degree west, (c) View from the north pole, (d) Altitude at different 
longitude. 
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One of the consequences of the fact that in its motion the same particle can cross a 
detector several times is that the measured flux of such upgoing and downgoing particles 
is the same. A build up of the measured rate, especially in the region close to the magnetic 
equator, connected with the multiple crossing of the detector plane rate, does not produce 
secondary particles unless the spiraling particle passes through and interacts with the 
atmosphere. The motion of the particle being determined by the Earth's magnetic field, 
negative particles travel along the same trajectories as the positive ones but in the opposite 
direction. 

The production of secondary pions and protons resulting from the interactions of 
cosmic (mostly He) nuclei with the atmosphere is treated using the superposition approx- 
imation [21]. 

The calculation presented below takes into account all effects connected with produc- 
tion, propagation, interaction and decay of all particles of the atmospheric shower. 

4 Particle flux 

The results of Monte Carlo simulation presented in the following correspond to a statistics 
of ~ 1.5 * lO^'' protons isotropically emitted from a sphere of 10 Earth's radii. This is 
equivalent to a 0.62 ps exposure. 

The dependence on the geographical coordinates of the rate of protons crossing a 
sphere surrounding the Earth at the altitude of 400 km is shown in Figure IHl for downward 
(top picture) and upward (bottom picture) protons. The rate includes all i.e. primary and 
secondary, protons and is averaged over all directions in the corresponding hemispheres. 

A typical picture with high rate near the magnetic poles and the depleted by the 
magnetic cutoff regions near the magnetic equator is clearly seen in the downward proton 
rate behaviour. The geographical pattern of upgoing protons is similar in shape but at 
much lower particle rate. 

The flux of protons going toward the Earth fPigslTUk and b) consists of original protons 
from space i.e. those with p/pmit >0. 9999996, where pmu is the initial cosmic proton 
momentum and p is the momentum of the proton crossing the detection sphere (see FigO}, 
protons scattered in the atmosphere (0.995< p/pmit <0. 9999996. Fig|H) and secondary 
protons {p/pinit <0.995. Figs ]5l6l7lHj) . The flux of upgoing protons fFigsITUh and d) 
contains reflected protons most of which have lost some energy in the atmosphere as well 
as scattered protons on their way from the Earth and the secondary protons. 
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Protons at 400 km 




Figure 9: Proton rate (in KHz/m^ sr) at 400 km altitude averaged over all directions. 
Top: earthward protons, bottom: spaceward protons. 
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Protons at 400 km 
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Figure 10: Partial (with respect to initial proton) momentum of protons detected at 400 
km. (a) All downgoing protons, (b) Close up on the close to unity region of (a), (c) All 
upgoing protons, (d) Close up on the close to unity region of (c). 
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The important feature of the primary flux relevant to neutrino production is that up 
to momenta as high as ~40 GeV it becomes anisotropic for directions close to parallel 
to the Earth's surface, (see FiglTT|l. This decrease is not in contradiction with the AMS 
measurement because the orientation of the AMS detector during the flight did not allow 
to measure the horizontal flux. 
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Figure 11: Dependence of primary proton flux at 400 km on the proton incidence angle 
with respect to nadir, (a) All momenta, (b) Solid line:p < 1.5 GeV/c, dashed line:1.5 < 
p < 5.0 GeV/c. (c) Solid line:5.0 <p< 10.0 GeV/c, dashed line:10.0 <p< 40.0 GeV/c. 
(d) p> 40. GeV/c. 
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Figure ^1 illustrates the behaviour of the flux of primary protons. Figure IT^ shows 
the dependence of the cutoff on the magnetic latitude. The cutoff momentum is about 
10 GeV/c at the equator and goes down to zero at the poles. One can see, however, that 
there are some under cutoff particles even in the flux of primary protons. These under 
cutoff particles are present at all magnetic latitudes (FigsIT21a and d). The other feature 
of these protons is that they are moving predominantly, and in the magnetic equator 
region exlusivly (Figsll2b and c), toward the Earth. 

The scattered protons fFig lT^ are mostly detected in the regions of the flux max- 
ima close to the magnetic poles (FiglTSk). These are mostly albedo protons (FigsfTSb 
and d) although there is a considerable amount of those (FigsEb and c) reentering the 
atmosphere (see figE)). 

Secondary protons (Fig|T3j) produced at the magnetic poles add up to the albedo 
flux (FiglT^) whereas those produced in the regions closer to the equator follow the 
trajectories illustrated in Figs ]5l6l7l IHl producing equal fluxes of upgoing and downgoing 
undercutoff particles fFiglT^). The impact points of particles with multiple crossing of 
the detection sphere are clearly seen in FiglT^. The rate of under cutoff protons in the 
magnetic equator region is dominated by the long traveling protons (see Figs l6|7j) . 

Positrons and electrons resulting from the decays of secondary pions display the be- 
haviour similar to that of secondary protons allowing for much faster energy losses in the 
atmosphere due to the difference in radiation and nuclear interaction lengths. 

5 Particles detected by zenith facing AMS 

To compare the results of calculations with the AMS measurements j24| ES] the overall 
fluxes produced in the simulation were restricted to the AMS acceptance during the zenith 
facing flight period. [23 IH^] • In accordance with the AMS construction no particle with 
kinetic energy below ~ 120 MeV can trigger AMS. The AMS orbit spans within the 
geographic latitude of ±51.7°. The corresponding cuts are applied on the MC simulated 
data. The result of Monte Carlo is an absolute one. No renormalization is applied to 
make a comparison with the data. As in AMS publications the spectra are presented 
as a function of kinetic energy of the particles. Figure ^1 shows the spectra of protons 
obtained in the simulation for different positions of AMS with respect to the magnetic 
equator. The result of simulation correctly reproduces the characteristic features present 
in the AMS data (black dots in the figure) such as: 

• the value of the cutoff at different magnetic latitudes; 

• the presence of protons with under cutoff momenta at all magnetic latitudes; 

• the equality of fluxes of under cutoff protons going to and from the Earth; 
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Figure 12: Primary protons detected at 400 km. Momentum at different magnetic latitude 
for (a) downgoing and (b) upgoing protons, (c) Average flux at different angles with 
respect to nadir. Solid line - all primary protons, dashed line - only under cutoff protons 
in the equator (|0m| <0.9) region, (d) Momentum spectra at different magnetic latitudes 
(Gm). Solid line -0. 9 < 0m <1.1, dashed line 0.7 < Gm <0.9,dotted line - 0.4 < Gm <0.7, 
dashed-dotted line -|Gm| <0.4 
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Figure 13: Scattered protons detected at 400 km. (a) Geographic coordinates of the 
crossing points, (b) Average flux at different angles with respect to nadir. Momenta 
dependence on magnetic latitude for (c)- downgoing and (d) - upgoing protons. 
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Figure 14: Secondary protons detected at 400 km. (a) Geographic coordinates of the cross- 
ing points (b) Avergae flux at different angles with respect to nadir for pole (|©m| >0.7 
rad, solid line) and equator (|0m| <0.7 rad., dashed line) regions, momentum dependence 
on magnetic latitude for (c)- downgoing and (d) - upgoing protons. 
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Protons in AMS. MC 
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Figure 15: Proton flux in AMS at different magnetic latitudes (|6m|)- Solid line - flux 
from the top, dashed line - flux from the bottom, a) |©m| <0.2 rad, b) 0.2< |©m| <0.5 
rad, c) 0.5< |©m| <0.8 rad, c) 0.8< \Qm\ <1-1 rad. The dots is the AMS measurement. 
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Positron flux in AMS. MC 

1 



a) 



slO 



10 




s 



10 



10 



Energy (GeV) 




Energy (GeV) 




Energy (GeV) 



Energy (GeV) 



Figure 16: Secondary positron flux in AMS at different magnetic latitudes (|0m|- Solid 
line - flux from the top, dashed line - flux from the bottom, a) \Qm\ <0.3 rad, b) 
0.3< IOmI <0.6 rad, c) 0.6< ISm] <0.8 rad, c) 0.8< \Qm\ <1-1 rad. The dots is the AMS 
measurement. The enhancement at higher energies of the measured flux at high magnetic 
latitude is due to primary cosmic positrons detected at the lower magnetic cutoff region. 
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Figure 17: Secondary electron flux in AMS at different magnetic latitudes (|0m|- Solid 
line - flux from the top, dashed line - flux from the bottom, a) |©m| <0.3 rad, b) 
0.3< IOmI <0.6 rad, c) 0.6< |Gm| <0.8 rad, d) 0.8< IGmI <1.1 rad. The dots is the 
AMS measurement. The enhancement at higher energies of the measured flux at high 
magnetic latitude is due to primary cosmic electrons detected at the lower magnetic 
cutoff region. 
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Positron/electron flux ratio in AMS. MC 
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Figure 18: Calculated dependence on magnetic latitude of positron/electron flux ratio. 
The dots is the AMS measurement. 
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Figure 19: Vertical muon flux at the sea level. Histrogram: this study. Circles: data from 
Ref . 01] , triangles: data from Ref . [IH] . 
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• the energy spectra of under cutoff protons and their absolute flux. 

Figures Uni and [T7| show the spectra of secondary positrons and electrons correspond- 
ingly. Although the positrons and electrons can not contribute to atmospheric neutrino 
production they are (together with cosmic muons) the "spectators" of neutrino produc- 
tion in the decays of particles in the atmospheric showers and consequently the correct 
prediction of the fluxes of these particles is a proof of validity of the whole simulation 
procedure including the neutrino flux predictions. The simulated positron and electron 
spectra are similar to those obtained in the AMS experiment [23] for the secondary leptons 
in terms of equality of fluxes of upgoing and downgoing particles and energy dependence 
of the flux at different magnetic latitudes. 

The difference in the measured and simulated secondary leptonic (and to some extent 
secondary proton) fluxes at low magnetic latitudes (FigslTHk. fT7k and fTHk ) is due to 
insufficient statistics of the present Monte Carlo study. As mentioned above, the measured 
rate in this region is largerly connected with the relatively rare long traveling particles 
multiply crossing the AMS orbit plane. Actually, the peak in the spectrum of figure 
ITBk is due to just 3 spiraling positrons each crossing the AMS detector plane about 
500 times in both upward and downward direction. Nevertheless the measured relative 
rate of secondary leptons [SHI HO] and their dependence on the magnetic latitude is well 
reproduced by the simulation (Fig lTK|) . It is worth to emphasize that the impact of long 
traveling secondaries on the production of atmospheric neutrinos is negligible. 

Apart from the data connected with those measured by AMS the present simulation 
includes a prediction of the fluxes of muons at the sea level. The histogram in FigJT^ 
shows a result obtained in this study for the vertical muon flux. The result is in good 
agreement with the measured fluxes reported in ^1 1^ . 

6 Atmospheric neutrino fluxes 

As for the fluxes of charged particles discussed in the previous section all results that follow 
are obtained directly without any renormalization. The dependence of the neutrino flux 
averaged over the entire Earth's surface on the neutrino direction with respect to nadir 
is presented in Figl^Uk. The overall equality of neutrino fluxes to and from the Earth is 
clearly seen. The dashed line in the figure is a mirror image of the flux from the top. A 
small increase of the flux of almost horizontal neutrinos coming from inside the Earth is 
due to neutrinos produced by the cosmic particle's showers in the Earth. The relative 
flux of different kinds of atmospheric neutrinos (i/^, j/g, t'e) is constant for all incidence 
angles (Figl2()b.). The averaged over all directions energy spectra of neutrino of different 
kind are presented in FigQUh. The dependence of the flux on the azimuth (with respect 
to east direction) angle fFigOUH) reflects some east-west asymmetry of average neutrino 
fluxes. 
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Overall neutrino flux 




Figure 20: Neutrino flux averaged over the Earth's surface. The angle of incidence (0) is 
measured from nadir (cos©=l for particles going vertically down). The dashed line in a) 
is a mirror image of the flux from the top. b) Relative flux of different kinds of neutrino, 
c) Energy spectra, d) Azimuth angle measured from direction toward east. Solid line - 
D^, dashed line - u^, dotted line - De, dashed-dotted line - Ug- 
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Neutrino flux map 




Figure 21: The geographical map of the average atmospheric neutrino flux in KHz/m^sr. 
Top: Downward doing. Bottom: Upward going. 
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The actual picture of neutrino flux distribution is very different from the average 
one. FigE] shows the geographic maps of neutrino fluxes measured in KHz/m^sr for 
downgoing (top) and upgoing (bottom) atmospheric neutrinos. The downward flux is 
very nonuniform and, not surprisingly, strongly correlated with the flux of cosmic protons 
(see figini). The downward neutrino flux changes from more than 15 KHz/m^sr at the 
magnetic poles down to less than 3 KHz/m^sr at the magnetic equator whereas the upward 
flux is almost uniform at the level of 6-7 KHz/m^sr over the entire Earth's surface. For 
different regions of the Earth not only the total flux of neutrinos is different but also the 
kinematical parameters of the neutrinos. 

To allow the comparison with previous calculations of Ref . ^H] we compute the neutrino 
event rate in three equal area regions according to the geomagnetic latitude, although in 
view of neutrino flux rapidly changing with magnetic latitude (see FigJ21|) such division 
appears to be rather crude. 

As in jT3] to compute the event rates the v fluxes have been convoluted with the 
neutrino cross section gl] , , |12 • The angle in the distributions rFigs l22l2;-il24l25l2(il27|) 

is the neutrino one with no experimental smearing or inefficiency included. Apparently, 
the results of the present study differ from previous calculations both in the energy spectra 
and the angular distributions of atmospheric neutrinos. The flux of sub-GeV neutrino 
predicted by this calculation depends much stronger on the geographical region following 
the intensity of the primary cosmic particle flux (FigEj). The down/up asymmetry of the 
flux is much stronger. There is an important increase in the horizontal flux emerging from 
the Earth. This flux is connected with neutrinos produced under the Earth's surface at 
a depth of several meters. A part of these neutrinos can be detected in the underground 
detectors. The actual amount of "underground" neutrinos detected in a given detector 
depends on the detector position depth. 

7 Atmospheric neutrino fluxes at major experimen- 
tal sites 

For the relatively small area corresponding to different experimental sites the statistics of 
the present study is high enough to make reliable conclusions for neutrinos with energies 
below ~1.5 GeV. The parameters describing the neutrino flux at the Kamioka site are 
calculated for an area within the geographical coordinates 35° ±12° A^, 142° ±12° E for an 
underground detector at a depth of 520 m (2700 mwe). The averaged over all directions 
differential energy spectra of different neutrinos and antineutrinos are given in FigQHl 
together with the results from other authors. The present study gives lower neutrino rates 
at higher energies. The fluxes of all kinds of neutrino are strongly down/up asymmetric 
with a considerable contribution of "underground" neutrinos in the horizontal direction 
(FigHnb)- The ratio of muon(z/^ ± to electron(z/e ± fluxes (FiglSHt) suggests a 
tendency to decrease for horizontal directions. 
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Figure 22: Average nadir angle distribution for e-like events for detectors located in 
positions on the Earth with magnetic latitude sin(0M)= [-0.2,0.2]. a) Ei,=0. 1-0.31 GeV, 
b) E^=0.31-1.0 GeV, c) E^=l. 0-3.1 GeV, d) E^=3. 1-10.0 GeV. From Ref . HHI : dashed line 
- ID calculation without geomagnetic effects; dotted line - ID calculation; dash-dotted 
line - 3D calculation. Solid line -this work. 
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Electron events. Intermediate region 




Figure 23: Average nadir angle distribution for e-like events for detectors located in 
positions on the Earth with magnetic latitude sin(9M)= [0.2,0.6]. a) Ej,=0. 1-0.31 GeV, 
b) E^=0.31-1.0 GeV, c) E^=l. 0-3.1 GeV, d) E^=3. 1-10.0 GeV. From Ref . HHI : dashed line 
- ID calculation without geomagnetic effects; dotted line - ID calculation; dash-dotted 
line - 3D calculation. Solid line -this work. 



30 




Figure 24: Average nadir angle distribution for e-like events for detectors located in 
positions on the Earth with magnetic latitude sin(9M)= [0.6,1.0]. a) Ej,=0. 1-0.31 GeV, 
b) E^=0.31-1.0 GeV, c) E^=l. 0-3.1 GeV, d) E^=3. 1-10.0 GeV. From Ref . HHI : dashed line 
- ID calculation without geomagnetic effects; dotted line - ID calculation; dash-dotted 
line - 3D calculation. Solid line -this work. 
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Muon events. Equator region 




Figure 25: Average nadir angle distribution for /x-like events for detectors located in 
positions on the Earth with magnetic latitude sin(9M)= [-0.2,0.2]. a) Ei,=0. 1-0.31 GeV, 
b) E^=0.31-1.0 GeV, c) E^=l. 0-3.1 GeV, d) E^=3. 1-10.0 GeV. From Ref . HHI : dashed line 
- ID calculation without geomagnetic effects; dotted line - ID calculation; dash-dotted 
line - 3D calculation. Solid line -this work. 
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Figure 26: Average nadir angle distribution for /x-like events for detectors located in 
positions on the Earth with magnetic latitude sin(0M)= [0.2,0.6]. a) Ej,=0. 1-0.31 GeV, 
b) E^=0.31-1.0 GeV, c) E^=l. 0-3.1 GeV, d) E^=3. 1-10.0 GeV. From Ref . HHI : dashed line 
- ID calculation without geomagnetic effects; dotted line - ID calculation; dash-dotted 
line - 3D calculation. Solid line -this work. 
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Muon events. Polar region 
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Figure 27: Average nadir angle distribution for /x-like events for detectors located in 
positions on the Earth with magnetic latitude sin(0M)= [0.6,1.0]. a) Ej,=0. 1-0.31 GeV, 
b) E^=0.31-1.0 GeV, c) E^=l. 0-3.1 GeV, d) E^=3. 1-10.0 GeV. From Ref . HHI : dashed line 
- ID calculation without geomagnetic effects; dotted line - ID calculation; dash-dotted 
line - 3D calculation. Solid line -this work. 



34 



Neutrinos at the Kamioka site 
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Figure 28: Differential energy spectra of atmosplieric neutrino (averaged over all direc- 
tions) at the Kamioka site, a) z/g, b) z/g, c) z/^, d) u^. Squares from Ref. [7], dots from 
Ref . [H] , triangles from Ref. . Histograms - this work. 
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Neutrinos at the Kamioka site 




Figure 29: Neutrino flux at the Kamioka site, a) Energy spectra, b) Nadir angle distri- 
bution, c) Flux ratio of ///e neutrinos, d) Azimuth angle distribution. In a),b) and d) 
solid line - i>^, dashed line - i/^, dotted line - i>e, dashed-dotted line - u^- 
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Figure 30: Neutrino flux at the Kamioka site. Nadir angle distribution for a) electron 
(i^e, j^e) and b) muon (i/^, neutrinos of different energy, d) Energy dependence of the 
ratio of the muon/electron fluxes. In a),b) and c) - dashed line 0.1< E <0.31 GeV, solid 
line 0.31< E <1.0 GeV, dotted line E >1.0 GeV. d) Energy dependence of the flux ratio 
of downgoing to upgoing muon (solid line) and electron (dashed line) neutrinos. 
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Neutrinos at the Soudan site 
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Figure 31: Differential energy spectra of atmospheric neutrino (averaged over all direc- 
tions) at the Saudan site, a) z/g, b) z/g, c) u^^ d) z/^. Squares from Ref. 0, dots from 
Ref. [HI, triangles from Ref.^l]. Histograms - this work. 
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Figure 32: Neutrino flux at the Soudan site, a) Energy spectra, b) Nadir angle distribu- 
tion. c)Flux ratio of ///e neutrinos, d) Azimuth angle distribution. In a),b) and d) solid 
line - i>a, dashed line - i/a, dotted line - i>e, dashed-dotted line - i/e- 
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Neutrinos at the Soudan site 
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Figure 33: Neutrino flux at the Soudan site. Nadir angle distribution for a) electron 
{ug, i/g) and b) muon (f^, v^) neutrinos of different energy, d) Energy dependence of the 
ratio of the muon/electron fluxes. In a),b) and c) - dashed line 0.1< E <0.31 GeV, solid 
line 0.31< E <1.0 GeV, dotted line E >1.0 GeV. d) Energy dependence of the flux ratio 
of downgoing to upgoing muon (solid line) and electron (dashed line) neutrinos. 
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Neutrinos at the Gran Sasso site 
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Figure 34: Differential energy spectra of atmospheric neutrino (averaged over all direc- 
tions) at the Gran Sasso site, a) z/g, b) z/g c) z/^, d) i>^. Dots from Ref.jH]. Histograms - 
this work. 
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Neutrinos at the Gran Sasso site 
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Figure 35: Neutrino flux at the Gran Sasso site, a) Energy spectra, b) Nadir angle 
distribution, c) Flux ratio of ///e neutrinos, d) Azimuth angle distribution. In a),b) and 
d) sohd line 



z/^, dashed line 



i/ju, dotted line 



Vp, dashed-dotted line - Vp 
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Neutrinos at the Gran Sasso site 




Figure 36: Neutrino flux at the Gran Sasso site. Nadir angle distribution for a) electron 
{ug, Vg) and b) muon (f^, v^) neutrinos of different energy, d) Energy dependence of the 
ratio of the muon/clectron fluxes. In a),b) and c) - dashed line 0.1< E <0.31 GeV, solid 
line 0.31< E <1.0 GeV, dotted line E >1.0 GeV. d) Energy dependence of the flux ratio 
of downgoing to upgoing muon (solid line) and electron (dashed line) neutrinos. 
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This systematic effect for neutrinos with the highest ffux (see Fig|29b) if not taken into 
account may lead to a conclusion about z/^, z>^ disappearance. The east-west asymmetry 
of electron and muon neutrino is visible in Figl^HH. 

The energy dependence of the angular distribution of the flux of electron and muon 
neutrino is illustrated by FigslHUk and EDb correspondingly. Note the absence of the 
increase in the muon neutrino flux in the horizontal direction. Figure IHIH i shows the 
energy dependence of the downward/upward flux ratio for muon (solid line) and electron 
(dashed line) neutrinos. The ratio is the same for two different kinds of neutrino in the 
energy range below 0.31 and above 1.5 GeV. However, for the region between 0.31 and 
1.5 GeV where the bulk of the experimental data is collected the down/up ratio for muon 
neutrinos is systematically higher than that for electron neutrinos. This systematic effect 
may fake muon neutrino disappearance. 

Neutrino fluxes at the Soudan and Gran Sasso sites are calculated for the areas with 
geographical coordinates of 48° ± 12° iV, 98° ± 12° W and 42° ± 12° iV, 13° ± 12° E 
and the depth underground of 700 m and 750 m correspondingly. As for the Kamioka 
site the averaged over all directions differential energy spectra of different neutrinos and 
antineutrinos at the Soudan (Fig]3T|l and Gran Sasso (Figl3l|l sites obtained in the present 
calculation give lower neutrino rates at higher energies than those obtained by other 
authors. The angular distributions of neutrino flux is strongly up/down asymmetric with 
predominance of the downward flux at the Soudan site fFiglH^) whereas it is almost 
up/down symmetric for Gran Sasso (FiglSSb)- The value and the energy dependence 
of the downward/upward flux ratio (Figsl33h)36h) for muon (solid line) and electron 
(dashed line) neutrinos is different for the two sites. At both experimental sites this 
ratio is almost the same for neutrinos of different kind with energies less than 1.5 GeV. 
There is no strong indication that the ratio of muon to electron fluxes at the Soudan site 
(Fig|22tI2Sli) depends on the neutrino direction. On the contrary the tendency for the 
ratio of muon to electron fluxes to decrease for directions close to a horizontal plane is 
clearly seen for neutrinos in the energy range 0.31 - 1.0 GeV in Gran Sasso (solid line in 
FigOnt). 



8 Summary and conclusions 

The proton flux near the Earth consists of primary protons from space, primary protons 
scattered in the atmosphere and secondary protons produced in atmospheric showers. 
Each component of the proton flux has different angular distribution. The results of the 
above Monte Carlo study of propagation in space and interaction with the atmosphere 
of cosmic particles show that the primary proton flux in the vicinity of the Earth is 
essentially anisotropic at the directions approaching the horizontal plane. This result is 
in contradiction with the assumption of most atmospheric neutrino flux calculations. The 
anisotropy has an important impact on the production of higher energy neutrino. 
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The momentum spectra of primary protons from space display at different latitudes 
the expected [221 ESI EZI behaviour connected with the Earth's magnetic field (cutoff). 
There exist, however, a non negligible amount of primary protons with momenta well 
below the magnetic cutoff. 

The momenta of the considerable number of scattered primary protons present at all 
magnetic latitudes are close to the cutoff value. When backtraced, the scattered protons 
have "forbidden" trajectories and consequently their contribution to atmospheric neutrino 
production was not taken into account in previous calculations of neutrino fluxes. 

Predominantly secondary under cutoff particles (protons, e"*", e~) are present in equal 
amount and the same momenta in the fluxes of particles going to and from the Earth. 
The simulation correctly describes the fluxes of protons and leptons (e"*", e~) measured by 

AMs 12a ESI mi. 

The atmospheric neutrinos produced in the simulation have the energy spectra, relative 
fluxes and angular distributions different from those obtained in the previous atmospheric 
neutrino flux calculations 13 IHl El UHl UH 1121 HH US] • The difference is a direct conse- 
quence of the anisotropy of the cosmic ray flux at the vicinity of the Earth. As it follows 
from the present study, the flux of primary cosmic particles at their entrance into the at- 
mosphere is diminishing when approaching the horizontal direction (see Fig ITT|) . Because 
of that, the fraction of high energy secondaries capable to decay before they lose most 
of their energy on the ionization of the atmosphere decreases. This results in smaller 
neutrino fluxes at higher energies. 

The up/down asymmetry of the neutrino fluxes resulting from this study is essentially 
different from the previous calculations. Similar to previous 3-D calculations the flux of 
atmospheric neutrinos has a maximum in the angular region around the horizontal plane. 
However, as it is demonstrated in this study there is an important contribution to the 
horizontal flux from neutrinos produced just under the Earth's surface. A part of the flux 
of the "underground" neutrinos can be detected in the underground experiments. The 
horizontal region is the most important for determination of the up/down asymmetry of 
the atmospheric neutrino fluxes and consequently the result of the present study may 
affect the eventual experimental measurement of the neutrino oscillation parameter Am^. 
Calculated for different experimental sites the angular distributions of neutrino fluxes 
of different kind and their dependence on the neutrino energy suggest the presence of 
effects faking disappearance of upward going muon neutrinos. These effects are different 
at different experimental sites (Figures IHHh.d: IHHh.dlHHfc.d). This could probably be the 
reason why different experiments do not agree in their estimate of oscillation parameters. 
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